Skip to content

Issues/improve jsonschema speed#1537

Merged
ikelos merged 5 commits intodevelopfrom
issues/improve-jsonschema-speed
Jan 20, 2025
Merged

Issues/improve jsonschema speed#1537
ikelos merged 5 commits intodevelopfrom
issues/improve-jsonschema-speed

Conversation

@ikelos
Copy link
Member

@ikelos ikelos commented Jan 9, 2025

This precompiles the schema file for jsonschema, meaning that (for some reason) it's already faster than the old combined method, but also if many json files are verified against the same schema there'll be a several second win each additional time (due to not having to recompile the schema).

I tried using fastjsonschema but the error messages are less descriptive and it fails validation for most of our existing handcoded and generated files, because the timestamps they generate don't include the timezone offset at the end (+XX:XX) finding all datetimes within a json file would be a pig (and slow) so better to change the validator somehow (still investigating) and/or figure out why jsonschema allows the old times and fastjsonschema does not...

Anyway, lemme know what you think and if the changes speed-up your testing workflow much?

@ikelos ikelos requested a review from atcuno January 9, 2025 18:23
@ikelos
Copy link
Member Author

ikelos commented Jan 18, 2025

Possibly enough to resolve #1535 , but I won't close it off in case this speed up isn't enough...

@atcuno could you give this a test please?

@ikelos ikelos merged commit d2314ed into develop Jan 20, 2025
24 checks passed
@ikelos ikelos deleted the issues/improve-jsonschema-speed branch January 20, 2025 08:26
@ikelos
Copy link
Member Author

ikelos commented Jan 25, 2025

Sorry, just trying to verify whether this sped things up enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants